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(57) Abstract 

A voice controlled voice-messaging system (1) 
permits manipulation of voice messages over the tele- 
phone without the need for keypressed commands. Ma- 
nipulation includes playback of messages, deletion of 
messages, saving messages, and sending a new message 
to a recipient from a user-defined list. Access to the sys- 
tem is by spoken password. Voice input also allows: en- 
rollment of a spoken password; granting or refusing us- 
er access to the system based on verification of the user 
speaking a password; recognition of spoken "yes" or 
"no" in response to prompts by the VMS ; recognition of 
"stop" to interrupt and stop a voice message during 
playback; enrollment of a spoken personal directory of 
names of potential message recipients ; and recognition 
of names in the spoken personal directory. The inven- 
tion uses speaker verification, speaker-independent re- 
cognition (SIR) of "yes/no", speaker-independent re- 
cognition of "stop" during message playback, and 
speaker-dependent recognition (SDR) of limited vocabu- 
lary of short phrases such as a list of message recipients' 
names. Spoken password processing comprises calibra- 
tion, enrollment and verification modes. 
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VOICE CONTROLLED 
MESSAGING SYSTEM AND PROCESSING METHOD 



Field of Invention 

This invention generally relates to digital voice processing methods for control of 
voice messaging systems. The invention specifically relates to hands-free or total voice 
control of voice processing systems connected to the telephone network. 

Proprietary Rights Notice 

A portion of the disclosure of this patent document contains material which is subject 
to copyright protection. The copyright owner has no objection to the facsimile reproduction 
by anyone of the patent document as it appears in the Patent & Trademark Office file or 
records, but otherwise reserves all other rights. 



Background of the Invention 

A. Voice Controlled Messaging Systems 

3Q In voice messaging systems (VMS) coupled to the public switched telephone network. 

a user (or "subscriber") usually controls the VMS by pressing push buttons by hand on a dual 
tone multi-frequency (DTMF) keypad of a telephone set. This method of data input is 
inconvenient, and even dangerous, when the user's hands, eyes, or both, are busy. 

For example, when a user is operating a car phone or cordless telephone the user may 

35 be occupied by other tasks (such as driving a car) which make manual data input difficult or 

dangerous. Other circumstances include use of the telephone while typing, using a computer 
terminal, or working at a workbench. Owners of rotary dial telephones cannot generate 
DTMF signals needed by most VMSs. Moreover, persons having impaired sight and person> 
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who lack hands or have other physical handicaps may find using telephone sets difficult or 
impossible. In all these cases, voice commands are the only convenient means of interacting 

with and controlling a VMS . 

General functions of VMSs are well known, as shown in U.S. Patents Nos. 4.352.807 
and 4, 371.752 (Matthews et al.) which disclose voice-store-and-forward systems. In most 
prior art systems, the user controls all or most of the functions of a VMS by manual input 
of DTMF digits. For example, in the Matthews et al. '807 patent, DTMF keypresses are 
required for some system functions such as enrollment of message recipients. In both the 
'807 and '752 patents, the VMS requires DTMF input and responds with "beep" sounds 
rather than digitized voice prompts. FIG. 16 of the '752 patent indicates that the '752 system 
requires DTMF digits for user identification. 

Prior attempts to automate VMSs have focused on elements of a system but fail to 
automate the entire system. For example, U.S. Patent No. 5.048,074 (Dugdale) simply 
replaces DTMF pushbuttons with foot switches. 

Text to speech (TTS) conversion is a known means for supplying a text or e-mail 
message to a caller, as exemplified by U.S. Patent Nos. 4,716,583 and 4,659,877. However, 
prior TTS systems have required use of DTMF digits to configure and operate the system. 
20 as shown in FIGS. 3a and 3b of the '583 patent. Similar systems, exemplified by U.S. Patent 

No. 4.996,707, enable conversion of a facsimile (fax) document into ASCII text for routing 
to a TTS system. This enables audible playback of a fax. However, the '707 and similar 
systems have all required entry of DTMF digits for control. 

Voice command systems with limited capabilities are also known, as exemplified by 
U.S. Patent No. 5,051,924. This system and others requires DTMF dial-up of a VMS rather 
than voice command access to messages in the VMS. 

Prior voice messaging systems also tend to require excessive computation resources, 
since in typical systems, a single digital signal processor (DSP) IC, the DSP must perform 
many voice processing functions besides hands free control. U.S. Patent 4.974. 191 is typical 
of computation-intensive voice response systems. In a typical VMS, very few DSP machine 
cycles are available just for voice control. Thus, those of skill in the art would appreciate 
an efficient implementation to allow other voice-band activity of significant computational cost 
to run concurrently. 

35 Another desirable feature is to have hands free processing available on all voice ports 

of a VMS so that any user can use hands free processing. Yet another desired feature is real 
time response to voice commands. The prior art fails to provide these features. For 
example, typical performance of the AT&T VMS. which is well known in the art. is 8.8 
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seconds to verify a spoken password. In contrast, one embodiment of the present invention 
has operated with response times of less than one-half second. Yes/no recognition has been 
measured at under 700 ms. 

Another disadvantage of the prior art is that performance parameters are not 
completely configurable, i.e., the parameters cannot be changed to other values while the 
messaging system is operational. This is a disadvantage since configurability can be used to 
optimize the parameters to the desired level of performance for the available processing power 
and to match characteristics of the location or site of the system. 

Those skilled in the an would also appreciate a totally voice controlled messaging 
system implemented on a general -purpose digital signal processor (DSP) which serves 
multiple channels of voice-band activity while using a maximum number of processor cycles 
for voice control processing. 

B. Speaker Verification 

Speaker verification methods are also known in the art, as exemplified by U.S. Patent 
No. 5,056,150. The general object of speaker verification is to establish a digitally stored 
template for a particular speaker uttering a selected, uninterrupted word ("feature extraction"), 
and then upon subsequent trials to estimate the confidence level associated with the same 
speaker uttering the same word ("pattern matching"). Feature extraction performs 
transformations on the speech signal to yield a template that represents the signals being 
compared. Pattern matching makes a comparison between a stored template and a template 
generated for an input signal, and yields numeric results about the proximity of the two 
templates. In both processes a primary goal is eliminating undue statistical variation among 
separate trials. Speech recognition also involves other discrimination tasks, but the present 
invention relates most directly to the closeness of match between the template and the new 
utterance. 

The prior art of speaker verification generally tread feature extraction and pattern 
. matching separately. In general, prior art methods do not relate to a combination of feature 
extraction and pattern matching, which combination is disclosed in the present invention. 
Moreover, in the present invention feature extraction is accomplished using smoothed group 
delay function (SGDS) and pattern matching for speaker variation using the hidden Markov 
model (HMM), a combination not known in the art. 

The central function of feature extraction is to transform a brief time frame of the 
speech signal into a feature vector. A straightforward method is to measure the average 
energy of the signal over a given time frame. The same process is repeated for all the time 
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frames of interest (such as the time needed to utter a phrase). A two-dimensional pattern is 
produced, which may be compared to a similarly-generated one. This time-energy method can 
discriminate between short and long phrases, or between speech and non-speech, but cannot 

5 recognize words or identify speakers. 

A better method of feature extraction is to separate the signal into frequency 
components ("spectral analysis"). This can be done with bandpass analog filters, or in a 
digital signal processor by the Fourier transform. Instead of a single value for each time slice 
as in time-energy analysis, spectral analysis yields either a set of amplitude envelopes for each 

10 frequency analyzed. The resulting template is like a topographic map, in which the goal is 

to match the location and height of peaks. 

Spectral analysis can discriminate words of a language, but is poor for discriminating 
between speakers. Further, it is strongly affected by passing the signal through a channel that 

15 does not have a "flat" frequency response, and is affected by noise, both of which are 

problems in telephony. 

"Cepstrum" analysis has been applied to signals containing echoes. Like Fourier 
analysis, it yields a spectral representation, but the independent variable is time difference 
(lag) instead of frequency. Its computation is approximately the same as two Fourier 

20 transforms and a nonlinear expansion. Its benefit is that the resulting lag spectrum, or 

cepstrum. may separate the effects of three or more sources of a difference in speech timbre, 
thereby enhancing discrimination among speakers. 

The group delay spectrum is a differently weighted but similarly derived form of 
spectral analysis, and is described in Itakura & UmezakL "Distance measure for speech 
recognition based on the smoothed group delay spectrum", IEEE Conf. on Acoustics, Speech 
and Signal Processing (ICASSP), IEEE, 1987, pp. 1257-1260. It can yield a more prominent 
set of features for matching. 

Performance of cepstrum and group delay can be varied by changing parameters. 

30 Manipulation of two scalar values "s" and "tau" of Equation (7) of Itakura et al. can 

reconfigure one into the other, or either into another spectrum. The effect is like tuning a 
piano. In the prior art conventional windowing techniques are known to reduce effects of 
sampling or finite interval selection. The terms "windowing" and "smoothing" are often used 
interchangeably. 

35 ^ the pr i or art hidden Markov modeling (HMM) is used to establish an assumption 

about the underlying behavior of a physical process. In HMM's the Baum-Welch, or 
,r forward-backward" method is the central part of a solution to the model, but to be complete, 
the remainder of a solution must be specified. Poritz, "Hidden Markov models: a guided 
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tour" ICASSP, IEEE, 1988, pp. 7-13, Section 7 describes use of the hidden Markov model 
(HMM) and the Baum-Welch method in general speech processing. As noted in Poritz Fig. 
6 and its accompanying text, use of the method must be preceded by selecting (either 
randomly or deterministically) initial seed values for the auxiliary function "Q". then 
application of Baum-Welch, then assessment of whether a critical point has satisfactorily been 

reached, then reiteration as needed. 

As is known in the art, hidden Markov models can be characterized by order and 
number of states. Usually an increased order coupled with an increased number of states 
increases the computational burden more than the product of the two. Those of skill in the 
art would find useful a processing system which provides a lesser processing load when the 
order and number of states are both increased. Prior speaker verification methods have 
been used with a single-microphone, e.g., for entry to a secure room. Those of skill in the 
art would appreciate a speaker verification method optimized for use with the telephone 
network and which can cope with variation in performance of telephone lines and 
microphones. 
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Summary of Invention 

20 Accordingly, this invention provides means and methods for total voice control (hands 

free control) of voice-messaging and voice-response systems, through means for manipulation 
of voice messages over the telephone without the need for keypressed commands. 
Manipulation includes playback of messages, deletion of messages, saving messages, and 
sending a new message to a recipient from a user-defined list. Access to the messaging 
system is by spoken password: Voice input also allows the following functions: enrollment 
of a spoken password; granting or refusing user access to the system based on verification 
of the user speaking a password; recognition of spoken utterances of "yes" and "no" in 
response to prompts by the VMS; recognition of "stop" to interrupt and stop a voice message 

30 during playback; enrollment of a spoken personal directory of names of potential message 

recipients; and recognition of names in the spoken personal directory. 

In the preferred embodiment, a user accesses the system by dialing its telephone 
number; thereafter, interaction of the user and the system is entirely controlled by voice, 
leaving the user's hands and eyes free for other tasks. The preferred embodiment uses four 

35 main speech technologies: speaker verification, speaker-independent recognition (SIR) of 

"yes/no", speaker-independent recognition of "stop" during message playback, and speaker- 
dependent recognition (SDR) of a limited vocabulary of short phrases such as a list of 
message recipients' names. The invention achieves real-time response for all four 
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technologies, is highly resistant to noise, is adept at rejection of invalid speech by impostors, 
and has robust error handling. 

The present invention also provides improved methods for automatically verifying the 

5 identity of a speaker seeking access to voicemail transactions by comparing a digitized. 

spoken phrase to a stored representation of that phrase by the same speaker. The methods 
applied comprise a calibration mode, an enrollment mode and a verification mode. The 
physical means for carrying out the method are one or more programmable processors, such 
as a digital signal processor hosted by a general-purpose computer, the latter controlling 

10 voicemail functions after verification has been established. According to the invention. 

smoothed group delay spectrum (SGDS) is used for establishment of a distance metric to 
compare an input utterance with a stored template. The template comprises a statistically 
distributed standard of utterances. In the invention methods based on the hidden Markov 

15 model (HMM) are used for summation of a scalar likelihood value which may be windowed 

to discern whether the speaker is valid or an impostor. The invention increases the dimension 
of the feature vector, and increases the order of the HMM transition matrix, both beyond the 
order ordinarily required to recognize or reconstruct speech for increasing the method's 
ability to discriminate among speakers. The invention also improves processing rates of noise 

20 calibration and comparison, and provides enhanced spectral discrimination between the useful 

components of speech and unvoiced consonants or non-speech. 

In one preferred embodiment, the voice messaging system includes telephone line 
interface modules which couple telephones to at least one voice port. The modules provide 
digitized voice data to a digital signal processor (DSP). The DSP processes digital voice data 

25 and changes characteristics of digitized voice signals under control of a program stored in a 

program store coupled to the DSP. All program parameters are configurable, enabling site- 
specific performance optimization. 

The invention can be further understood with reference to the attached drawings: 
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Brief Description of Drawings 

FIG. L comprising FIGs. 1 A to ID, is a block diagram of a voice messaging system 
on which the present invention can be implemented; 
5 FIG. 1A is a system level block diagram of the voice messaging system; 

FIG. IB is a block diagram of a line interface controller in the system of FIG. 1 A; 

FIG. 1C is a block diagram of a preferred data format used by the system of FIG. 1 : 

FIG. 2 is a high-level flow diagram of principal features of a preferred embodiment 
of the invention; 

10 FIG. 3 is a block flow diagram of spoken password feature options in an alternate 

embodiment of the invention; 

FIGs. 4a to 4e and FIGs. 5a to 5d are state diagrams of logical flow in the 

embodiment of FIG . 3 ; 

FIG. 6a to 9B are block flow diagrams of logical flow of a control program 
representing one preferred embodiment of the invention, and in particular, 

FIGs. 6a to 6b are flow diagrams of an enroll password mode and an enroll recipient's 
name and word STOP mode of the preferred embodiment of the invention; 

FIGs. 7a to 7d are flow diagrams of a verify password mode and a review new 
20 message mode of the preferred embodiment of the invention, including a message playback 

and STOP recognition mode and a YES/NO recognition mode of the preferred embodiment 
of the invention; 

FIGs. 8a to 8b are flow diagrams of a review saved message mode of the preferred 
embodiment of the invention; and 

25 

FIGs. 8c ? 8d ? 9a, and 9b are flow diagrams of a send message mode including a 
recognize recipient name subroutine of the preferred embodiment of the invention. 
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Detailed Description of Prefe rred Embodiments 

In the following detailed description of the preferred embodiments, specific 
terminology is used for the sake of clarity. However, the invention is not limited to the 
5 specific terms used, but includes all technical equivalents functioning in a substantially similar 

manner to accomplish a substantially similar result. One of skill in the art to whom this 
disclosure is directed is assumed to be proficient in programming in the C language and the 
UNIX operating system, and to be familiar with B.W. Kemighan et al .. "The C Programming 
Language," ISBN 0-13-110163-3, and "UNIX Programming." Familiarity with "UNIVOX 
10 VDS General Description" and "UNIVOX Maintenance and Administration Manual". 

commercially available from Digital Sound Corporation, is also assumed. One of skill in the 
art should also be familiar with AT&T System V/386 User's Reference Manual. System 
Administrator Reference Manual and Programmer's Reference Manual, all commercially 
available from AT&T. 



15 



A. System Overview 

FIG. la shows a prior art voice messaging system 1 in which the present invention 
is preferably implemented. The system comprises control elements 10, telephone line 

20 interface elements 20, and peripheral interface elements 30, 40. These elements exchange 

data and control signals on a Multibus 50 which can follow the Multibus protocol developed 
by Intel Corporation. Multibus is a registered trademark of Intel. An independent bus 60, 
called a time division multiplexed (TDM) highway, enables fast transfer of digitized voice 
band data. The system 1 preferably is the VoiceServer 21 10 product commercially available 

25 from Digital Sound Corp., Carpinteria. California. This system is described in VoiceServer 

System Reference Manual, available from Digital Sound Corporation. 

The control elements 10 include a system controller 12, which preferably is an Intel 
386-class CPU with conventional support electronics, coupled to the Multibus and to asystem 

30 console 14, containing a cathode ray tube (CRT) display and keyboard entry system of 

conventional type. 

The telephone interface elements 20 include one or more analog line interface modules 
24, which receive incoming calls on a public switched telephone line 70. As is known in the 
art, the analog interface modules digitize incoming call signals and assign the call to a channel 
35 in the system. If incoming digital telephone lines are available, such as lines following the 

Tl standard protocol, then one or more Tl line interface modules 26 of conventional design 
are provided to couple Tl lines to the digital elements of the system. As is known in the art. 
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Tl interfaces provide digital voice communication at 1.544 megabits/second as defined in 
CCITT recommendations G.703 and G.704. 

Digital signal processing of voice messages and control signals is done by one or more 

5 line interface controllers (LICs) 22 which are shown in detail in FIGs. IB and 1C. Referring 

to FIG. IB, each LIC 22 preferably includes a complete conventional microcomputer 52 
coupled to a plurality of arithmetic processing units (APUs) 58a-58c each containing at least 
one digital signal processor (DSP). The microcomputer preferably is type 80386 
commercially available from Intel. The DSP preferably is the TMS320C25 available from 

10 Texas Instruments Inc.. In one preferred embodiment the microcomputer 52 is coupled to 

cache memory processor and a floating point processor 53 over 32-bit data and address buses. 
Output from the cache and floating point processors 53 are placed on 32-bit floating point 
(FP) data and address buses 53' and 53" which are coupled to a direct memory access 

15 controller (DMAC) 54. Also coupled to the FP buses are up to 8 mbytes of dynamic random 

access memory (DRAM) 55, and preferably eight of the APUs. The DMAC preferably is 
a type 82350 integrated circuit commercially available from Intel. A bidirectional Multibus 
interface 56 couples the LIC FP buses to the Multibus, by multiplexing and connecting signals 
from the buses to each other. A data bus two-way transceiver 59, reduces the 32-bit FP data 

20 bus to a 16-bit bus 59' which is coupled to a read-only-memory (ROM) 64 and control 

registers 65. The transceiver is conventional and enables two-way communication of a 16-bit 
signal on either a 16-bit or a 32-bit data bus. A single block 62 represents several common 
control circuits for bus arbitration, address decoding, TDM highway watchdog timing, and 
decoding requests for access to on-board registers. A TDM interface 63 couples the FP 

25 address bus and the 16-bit data bus to TDM highway 60. The TDM interface decodes a 

serial data stream on the TDM highway and assigns serial bit blocks in the stream to one of 
the APUs on the FP bus. Two buses, in the form of TDM highway 60 and the FP buses, 
enable separation of voice data from control functions by providing a separate bus. the TDM 

30 highway, dedicated to digitized voice data. These components and their interaction are well 

known. 

An exemplary APU is shown in block form in FIG. 1C. Each APU contains a DSP 
71, preferably the TMS320C25 of Texas Instruments Inc., coupled to a conventional memory 
having at least several kilobytes of storage in the form of a conventional electronic random 
35 access memory (RAM) 73. The RAM 73 preferably is divided into a program store 72 of 

preferably 32k words and associated program buffers 73, and a data store 74 (64k words) and 
data buffers 75. Control circuits 76P. 76D handle data interchange between the DSP and 
the memory. FP bus access is handled using registers 77. 78 and IDROM 81 . A segment 
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register (SEG REG) 78 extends 16-bit addresses from the DSP to 32-bit addresses used on 
the rest of the board. A command status register (CSR) 79 couples i/o interrupts to and from 
the DSP, and also provides reset and hold bits for the DSP. A time slot assignment (TSA) 
register controls orientation of TDM highway serial data. The IDROM contains character 
data to identify the APU board so that the system controller can determine the system 
configuration by reading the IDROM. The DSP preferably serves multiple channels of 
voice-band activity. The invention is preferably implemented in assembly code software on 
the DSP, as discussed in detail below. 

Depending on the needs of the system user, a plurality of peripheral devices are 
interfaced to the system. For example, a Small Computer System Interface (SCSI) host 
adapter 32 are coupled to the Multibus and a streaming tape drive 36, a floppy disk drive 34. 
and one or more mass storage devices such as hard disk drives 38 can also be connected in 
known manner. The hard disk drives provide primary storage for voice data and can also 
provide storage for system software; via the Multibus, the disk drives are indirectly coupled 
to the DSPs on the LICs. Further, a magnetic tape controller board 42 is provided to 
interface the Multibus to a streaming tape drive 43. A serial communication interface board 
44 can connect to a plurality of serial devices such as IODM 44A. modem 44B. printer 44C. 
and user ports 44D. Additional communications is provided using Ethernet board 46 and an 
X.25 board 48. Electronic and interface details of the elements designated 30 to 48 are 
conventional and well known. 



B- DSP Operation 

Each DSP in an APU on a LIC communicates with voice signals in a plurality of 
channels of data on the TDM highway. For each channel, at intervals equal to a 
predetermined sampling period (typically 8 KHz), the DSP receives a voice data sample value 
(receive data), and transmits a sample value (transmit data). Data reception and transmission 
occurs on the TDM highway. 

As shown in FIG. ID, voice data samples are obtained by the analog interface 
modules 24 which receive analog voice audio 80 from a telephone line on trunk 70. The 
audio is fed to an analog-to-digital converter 82 on the LIM 24 which samples the analog data 
at 8 Khz, the standard voice-band sampling frequency used in the telephone network. The 
samples are in linear 12-bit form, thus having a value represented by an integer ranging from 
-2048 to +2048. The sampled data is fed in a stream 84 from the LIM to the TDM 
highway. 
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By communicating through an APU on a LIC, the DSP receives and accumulates, over 
a fixed time period, a fixed number of receive digital data points 88 to form a receive 
"frame" 86 stored in a discrete area in the data store 74. Outgoing transmit data points are 
likewise accumulated to form a transmit frame in the data store. This period is called the 
frame duration and preferably comprises 180 sample points (88) representing 22.5 ms of 
voice data sampled from an audio signal. 

Over a frame duration, the DSP processes the signals represented by the received 
frames and transmit frames, for each channel of activity, as directed by DSP software in the 
program store 72. The DSP can perform several different types of processing including 
speech encoding and decoding, companding, tone detection and generation, speech 
recognition, text-to-speech conversion, etc. All require DSP processing or computation. 
Thus, the frame duration determines the maximum total number of computations possible per 
frame of transmit and receive data. 

The invention is preferably implemented on the system of FIG. 1 A using a computer 
program for the DSP written in the C source language. Preferably the C code is tested, 
debugged, and then hand-assembled into DSP assembler code, which is linked by the 
TMS302C2x DSP Assembler program commercially available from Texas Instruments. The 
assembled object code is stored on the hard disk drives 38 and loaded into DSP memory when 
the system is powered-up. Ordinarily the system remains on continuously so the software is 
always available in memory. Publications disclosing such DSP programming include 
"Second-Generation TMS320 User's Guide," No. 1604907-9702 (June 1989) and 
t TMS320Clx/TMS320C2x Assembly Language Tools User's Guide." No. 1604908-9706. 
commercially available from Technical Publications Manager, Texas Instruments Inc., P.O. 
Box 1443, MS 640, Houston, TX 77001, the contents of which are hereby incorporated by 
reference. 

C. Total Voice Controlled Messag in g System -- Gen eral Features 

The present invention provides a plurality of processing methods which are preferably 
combined to form a total voice controlled VMS. In a preferred embodiment, the invention 
preferably comprises a plurality of stored programs written in the C source language and 
stored in the disk drive 38. The programs can be combined into a single application program 
35 or can serve as stand-alone functions for selective integration into an application program. 

For example, the programs can be integrated into the InfoMail application program 
commercially available from Digital Sound Corp. (address above). InfoMail is described in 
the InfoMail User Interface Reference Guide, Digital Sound part no. 5950-0863, shown in 
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1 Appendix A and incorporated herein by reference. One preferred embodiment of the present 

invention is the C language program "handfreeB.c." shown in Appendices B and E hereto. 
Further information about the speaker verification feature of the invention is given in 

5 "VoiceServer System VDS Speaker Verification Application Guide," Digital Sound part no. 

5950-1203, shown in Appendix C hereto. Speech recognition processing is described in 
"VoiceServer System Speech Recognition Programming Reference," Digital Sound Corp. part 

no. 5950-1449. 

FIG. 2 shows the general logical flow of principal features in the method of the 
10 invention. Implementation of each feature is discussed in detail below and is more fully 

described in Appendix B. In general, the invention provides means for conducting an 
interactive dialogue with a user ("Subscriber") of a voice messaging system. Initiation of the 
dialogue (not shown) begins when at block 220 a user calls a voice messaging system 
("VMS") using the public switched telephone network. For example, a user of a VoiceServer 
2110 can call a telephone number designated as the "mailbox" of the subscriber. The VMS 
responds by playing back a prerecorded greeting. 

In one embodiment, enrollment functions are provided off-line or in a separate mode, 
as shown by blocks 234 to 240. In step 234, a subscriber is prompted whether to enroll a 
20 new password. Blocks 234 and 238 expect a YES or NO response from the subscriber. If 

affirmative then in block 236 the VMS permits enrollment (encoded recording of a new 
password. In blocks 238 and 240 enrollment of a message recipient's name is handled. 
These features are described in detail below. 

Next in block 222 the VMS plays a prerecorded prompt message (termed a "prompt"). 
25 which asks the subscriber whether any messages should be reviewed, as shown in block 222 . 

If not, then the VMS ends the session by branching to another DSP program or disconnecting 
the subscriber, as shown by block 223. If so, then the VMS plays a prompt which asks the 
user to speak his password, into the telephone. This prompt message, and others use by the 
VMS, preferably are each digitally stored in stored in separate digitized voice data prompt. 
The prompts are used in the embodiment of Appendix B. as described below. As shown in 
block 200 of FIG. 2. the VMS then waits while the user speaks and verifies that the password 
is correct. 

Speaker verification is a technology which allows a VMS to verify a speaker's 
35 identity, thus rejecting impostors. In a VMS, speaker verification can be used to establish 

a "spoken password". To gain entry to a voice mailbox, a user speaks a voice password. 
If the spoken password matches a prerecorded password enrollment template, the subscriber's 
identity is verified. Otherwise, the password is rejected and entry is denied. 
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Speaker verification is preferably implemented using a template matching method. A 
digitized speech signal is converted into a plurality of cepstral coefficients which are numbers 
uniquely identifying the speech signal. Hidden Markov models (HMMs) are used to represent 
speech and serve as a template. During enrollment an HMM is created. During verification 
the enrolled HMM is compared to an HMM created from an input utterance. 

If the password is not verified, the prompt is repeated and the user can try again. 
After two failed attempts, the VMS ends the dialogue and disconnects the phone connection. 

Otherwise, in step 201 the VMS prompts the subscriber whether new messages should 
be played back. The VMS waits for a YES or NO response f rom the subscriber. If YES 
is received, the VMS next begins to play back all new messages of the subscriber, as shown 
in step 202. Playback can be prefaced with a prerecorded message such as "you have three 
new messages." The messages are played back serially from the mass storage system. 
During playback, as shown in block 203. In response, the VMS will stop playback. 

Ordinarily the messages are digitized voice messages stored on the mass storage 
system. However, the present invention also provides the capability of playing back 
electronic mail ("e-mail") messages which are stored as text rather than digitized speech. E- 
mail messages are typically stored in a disk file in ASCII or EBCDIC character text. When 
20 an e-mail message is in the subscriber's queue, then the VMS plays a prompt asking whether 

the message should be delivered verbally. If so, then the message is retrieved and routed to 
a text-to-speech converter which synthesizes audible speech corresponding to the text of the 
e-mail message. If not, then the VMS plays a prompt asking whether the message should be 
delivered to a facsimile ("fax") machine. If so, then a separate dialogue, explained below, 
is initiated to elicit information about where to send a fax. The e-mail message is then 
retrieved and converted to fax format prior to sending. 

After playback is complete, the VMS plays a prompt asking whether the subscriber 
wishes to save any of the messages just played, as shown in step 204. If so. message saving 
30 is processed in step 206. 

Thereafter the VMS plays a prompt asking whether the subscriber wishes to delete any 
of the messages, as shown in step 208. If so, message deletion is processed in step 210. The 
VMS next plays a prompt asking whether the subscriber wishes to retrieve and replay any of 
the messages, as shown in step 212. If so, the message is retrieved and replayed as shown 

35 in step 213. 

During each of steps 204, 208, and 212. the VMS uses speaker-dependent recognition 
to process and interpret the subscriber's reply to the VMS prompt. In alternative 
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embodiments speaker-dependent recognition or other speech recognition technologies can be 
used. 

Steps 226 to 230 provide a send message feature. As shown in step 226, the VMS 
5 plays a prompt asking whether the subscriber wishes to send a message to another subscriber. 

If so, then in step 228 the VMS prompts the sending subscriber to speak the receiving 
subscriber's name. The VMS records the spoken name and compares it to a table of enrolled 
recipient names stored in the data s tore or on the mass storage device. If a match is made 
then in step 230 the VMS prompts the sending subscriber to speak a message which is 
10 recorded and then sent to the receiving subscriber's mailbox. Step 230 preferably includes 

substeps enabling the sending subscriber to replay, review, and erase the recorded message 
before sending it. 

The send message feature also preferably includes sending a message to a list of 
recipients in a table stored in the VMS. Alternatively, the VMS can use SDR to recognize 
the name of the message recipient and retrieve a mailbox address corresponding to the 
recipient from memory in the VMS. In such a case, the VMS preferably carries out a name 
confirmation step, by playing back synthesized speech of the name using name to speech 
(NTS) methods. The subscriber then confirms that the named recipient is correct with a 
20 "yes'V'no" response. Preferably the VMS also includes an "urgent" marking feature whereby 

messages sent to recipients can be marked as "urgent," placing the marked message at the 
beginning of the playback queue. 

Next the VMS plays a prompt asking whether the subscriber wishes to hear more 
messages processing options as shown in step 214. In step 214. 204, and 208. the VMS 
expects a "yes" or "no" answer which is interpreted, as explained below, using speech 
recognition of an affirmative or negative response. In the preferred embodiment the words 
YES and NO are sensed, but affirmative and negative responses in other languages can be 
sensed. If the response is yes. prompts are played back as shown in step 216. After the user 
selects an option it is processed in step 218. Thereafter the messaging session terminates. 

Next the VMS plays a prompt asking whether the subscriber wishes to hear more 
message processing options as shown in step 214. The VMS expects a "yes" or "no" answer 
which is interpreted, as explained below, using speaker independent recognition (SIR) of the 
word yes or no. If the response is yes, prompts are played back as shown in step 216. After 
35 the user selects an option it is processed in step 218. Thereafter the messaging session 

terminates. The options processed in step 218 include sending a new message to a specified 
mailbox. The options also preferably include sending a message to a list of recipients in a 
table stored in the VMS. Alternatively, the VMS can use SDR to recognize the name of the 
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message recipient and retrieve a mailbox address corresponding to the recipient from memory 
in the VMS. In such a case, the VMS preferably carries out a name confirmation step, by 
playing back synthesized speech of the name using name to speech (NTS) methods. The 
subscriber then confirms that the named recipient is correct with a "yes'Vno" response. 
Preferably the VMS also includes an "urgent" marking feature whereby messages sent to 
recipients can be marked as "urgent," placing the marked message at the beginning of the 
playback queue. 

Another option preferably processed in step 218 is administrative functions such as 
changing a mailbox configuration, user name, telephone or fax numbers, etc. 

When a message is routed to a fax machine then a separate dialogue is initiated to 
retrieve information about the destination of the message. The dialogue is preferably 
structured as a series of prompts which require yes/no answers. The dialogue first asks 
whether the fax should be routed to the subscriber's voice line, i.e., the line on which the 
dialogue is being carried out. If so, then the subscriber is told to hang up after which the 
VMS calls the subscriber's line and delivers the message by fax. If not, and if the subscriber 
has a separate fax line, then the subscriber is asked whether the message should be delivered 
to the subscriber's default fax line. If so, then the subscriber is told to hang up after which 
20 the VMS calls the subscriber's line and delivers the message by fax. If not, then the 

subscriber is prompted to enter a fax line number either using DTMF digits or by spoken 
digits. In the latter case, the VMS uses continuous SIR of the spoken digits to interpret the 
phone number to which the fax should be sent. 

25 

D. Detailed Description of Preferred Embodiment 

The preferred program of Appendix B operates according to the logical flow of FIGs. 
6a to 9b. Referring first to FIG. 6a, the method begins when the DSP enters block 630, 
which is the DSP processes at the beginning of a new call from a subscriber to the VMS. 

30 Start block 630 preferably comprises functions such as port initialization, data structure 

initialization, threshold initialization, and template loading as expressed in the statements on 
pages 1 to 5 of Appendix B. Thereafter the DSP tests, in block 638, whether the subscriber 
wishes to enroll a new password. Block 638 preferably uses the code of the "enrollPassword" 
function of Appendix B. If the test of block 638 is true then the DSP enters an enroll 

35 password mode 632. If false, then in block 640 the DSP tests whether the subscriber wishes 

to enroll a new message recipient's name and the word "stop" for use in message playback. 
If the test of block 640 is true then the DSP enters an enroll recipient's name and word stop 
mode 634 of FIG. 6a. If false then control passes to block 642, in which the DSP tests 
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1 whether the subscriber wishes to review (i.e. playback) stored messages from other 

subscribers. If true then the DSP enters a review message mode 636 of FIG. 7a. If false 
then control passes to block 644 in which the DSP performs further voice mail functions or 

5 disconnects the subscriber. 

1 Enroll P? *<:word Mode 632 
a_ General 

Enrollment is a process used to train the VMS to recognize the subscriber's voice and 
10 togeneratethesubscriber'sreferenceHMMCtemplate). Thus enrollment comprises recording 

a digital representation of features of a representative digitized utterance called a template. 
In Ae preferred embodiment of the present invention, the enrollment process is automated so 
that no DTMF entry is needed to create a password enrollment template. According to the 
invention, the accomplish enrollment, the VMS plays a prompt (such as "speakpw.V in the 
lD above list) to tell the subscriber to speak a password. In one preferred embodiment, the VMS 

prompt instructs the subscriber to speak the password a plurality of times, each time after a 
tone. The VMS waits for the subscriber to speak a password and detects the end of the 
password by sensing silence. The VMS then immediately generates the next tone and 

20 continues. 

Enrollment is preferably accomplished using a self-testing procedure with the logical 
flow of FIG. 6a and 6b. Self-testing occurs during enrollment and results in guaranteed 
verification of the password under similar line conditions. In one preferred embodiment, the 
VMS requires the subscriber to speak the password eight times. The first five are used to 
25 create the template, and the last three are used to test the template. The testing is performed 

using the same general computational steps used when a subscriber is trying to access a 
mailbox ("mailbox login"). However, the testing is done at a higher "confidence level" than 
that used during mailbox login. A higher confidence level means that a tighter matching 
criteria is applied by the system, resulting in a lower acceptance rate for impostors. By using 
a higher confidence level during enrollment testing, the VMS effectively "teaches" the 
subscriber how to speak the password consistently so that after enrollment, the subscriber will 
always gain entry to a desired mailbox. This also improves the security value of the 
password. 

35 Preferably all important system variables and parameters are re-configurable. These 

variables include (1) the number of entries used to make the template, (2) the number of 
entries used to test the template. (3) the confidence level to be used during enrollment 
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template testing, and (4) the confidence level used for password verification during mailbox 
login. 

One of skill in the art will recognize that increasing the value of variable (1) causes 
the template to be harder to match. Decreasing the values of variables (1) and (2) shortens 
the enrollment procedure. Increasing the value of variable (3) causes template creation to be 
more difficult, but the resulting template is also harder to impersonate. Increasing the value 
of variable (4) makes it harder for impostors to match the template, but also makes it harder 
for a subscriber to complete mailbox login. 

Preferably the VMS is provided with means for giving interactive assistance to a 
subscriber during enrollment. If the subscriber is having difficulty creating a valid template, 
assistance is offered. Preferably four types of assistance are given. If the user fails to 
successfully match a template just created, then the system plays back messages advising the 
subscriber how to improve consistency. If the subscriber waited too long before speaking, 
e.g. more than five seconds, a warning message is played back. If the subscriber spoke a 
password which is too long, another warning message is given. If the subscriber pressed a 
DTMF button or caused "talk-off" (spoken sounds which sound like DTMF tones), a third 
type of warning message is played. 
20 To implement subscriber assistance, the method of the invention provides several 

different error conditions. For example, the invention can report utterance duration and 
confidence level, and also indicate "utterance too short," "utterance too long." "not loud 
enough," "bad match," and "non speech." 

In the preferred embodiment, the VMS is provided with a plurality of verbal pre- 
recorded voice prompts, stored on the disk drive 38. Each prompt is indexed with either a 
file name or a prompt number, and can be replayed by the VMS under control of a stored 
program. A preferred set of prompts is shown in Tables 1(a), 1(b), 1(c), 2(a). and 2(b): 

Table 1(a) — Voice Prompt Set 
30 File Name Content of Prompt 

calibrate. v "Please remain silent after the tone." 

erase . v "Do you want to erase this message? Please say yes or no after 

the tone." 

firstnew.v "First ; new message. From — " 

first save.v "First ; saved message. From — " 

from.v "From—" 

35 header l.v [Person l's name] 

header2.v [Person 2's name] 

header3.v [Person 3's name] 

message l.v [Person l's message] 

message2.v [Person 2's message] 
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message 3.v 

messerase.v 

messsave.v 

msgsent.v 

namereject.v 

number.v 

password .v 

ready .v 

recipname.v 

recmsg.v 

reject.v 

repeat.v 

review. v 

reviewmsg.v 

reviewsave.v 

sendl.v 
send2.v 
send3.v 
sendmessage.v 



sensmsg.v 

silent.v 

speakname.v 

speakpw.v 

stop.v 

timeout .v 

name00-name09 

word00-word09 
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[Person 3's message] 
"Message erased." 
"Message saved." 
"Message sent." 

"The recipient's name has not been recognized. 

"You have two new messages and one saved message." 

"Please speak your password after each tone." 

"Please say YES when you are ready to begin enrollment." 

"Please say the recipient's name after the tone." 

"Please record your message after the tone." 

"Your password has been rejected. Goodbye." 

"Your voice has not been recognized. Please say yes or no 

again." 

"Do you want to review your messages? Please say yes or 
no at the sound of the tone." 

"Do you wish to review your message? Please say yes or no 
after the tone." 

"Do you wish to review your saved messages? Please say 
yes or no after the tone." 
"Sent at 9:15 am." 
"Sent at 9:55 am." 
"Sent at 10:23 am." 

"Do you want to send a message? Please say yes or no after 

the tone. " 

"Send a message?" 

"Do you wish to send your message? Please say yes or no 

after the tone." 

[silence] 

"After each tone, speak the name—" 
"Please speak your password after the tone." 
"Please say STOP after each tone." 
"Goodbye." 

Names spoken to identify air individual (as in voice mail). 
Names spoken as they are to be entered into the recog 
template. 



Tnhle lfb) - Spoken Password Enrollment Prompts 
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Prompt No. 

0044 
0099 



Content of Prompt 
Recording ERASED. 

You have entered too many invalid parameters. 
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0797 
0798 

0801 

0920 
0921 
0980 
0982 



30 



0989 



01485 
01484 
01486 

01487 

01488 
01489 
01490 
01491 
01492 

01493 



01507 



To enter a NUMERIC password, press ONE. To record a 
SPOKEN password, press TWO. 

To establish a spoken password you need to record your 
password several times. Please SPEAK your password in 
your normal voice keeping it the same each time. To 
CANCEL all recordings and start over, press STAR. 
Your spoken password has been accepted, and is now 
enabled. 

You must now record your password... 
...times. Please SPEAK your password after EACH tone. 
The last recording was too LONG. Please try again. 
Your new spoken password has not been recognized by 
Voice Mail. Try using a word with stronger syllables. Or 
pronounce your spoken password more clearly. If your 
spoken password is a phrase, leave no silence between 
words. 

Passwords MUST be between ONE and TWO seconds in 

length. A word or phrase with four strong syllables works 

well. Do not leave any silence between password phrases. 

To ENABLE your spoken password, press ONE. 

To DISABLE your spoken password, press TWO. 

To record a NEW spoken password, press FIVE and follow 

the instructions. 

To record a spoken password, press FIVE and follow the 
instructions. 

Your spoken password is now ENABLED. 

Your spoken password will remain ENABLED. 

Your spoken password is now DISABLED. 

Your spoken password will remain DISABLED. 

A spoken password cannot be recorded at this time due to 

system failure. 

You may not press keys during this process. Just speak your 
password after each tone, 'and wait for the next tone or 
prompt. 

Your ORIGINAL spoken password is still ENABLED. 



Table 1(c) — Spoken Password Mailbox Login Prompts 
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Prompt No. 
0100 



Content of Prompt 

Please dial your password and press POUND. 
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Please SPEAK your password, after the tone. Or DIAL your 
password and press POUND. 
mr7 Please dial your MAILBOX NUMBER and press POUND. 

0127 or press STAR to CANCEL and RE-DIAL your 

PASSWORD. UATTRnY 
Your PASSWORD does NOT match your MAILBUX 

NUMBER. 

Please enter your mailbox EXTENSION number and press 
POUND. Or press STAR to CANCEL and RE-DIAL your 
mailbox ID number. 

Your PASSWORD does not match your mailbox ID or 
mailbox extension number. 



0757 



20 0225 



Table 2fa~> - Standar d Message 3001 

Msg No- Content 

Q223 To change your password, press ONE. 

0224 To set up or remove a guest mailbox, press TWO. 

0225 To change instruction length, press THREE. 

0226 To select outcall options, press FOUR. 

0227 To select paging options, press FIVE. 



Table 2(b) - Brief Mess age 3001 
Msg No. Content 

0218 For password, press ONE. 

0219 For guest mailboxes, press TWO. 

0220 For instruction length, press THREE. 

0221 For outcall options, press FOUR. 

0222 For paging options, press FIVE. 

30 Logical flow of the enroll password mode 632 is shown in detail in FIG. 6a 

and 6b. Preferably the mode is implemented using the function "enrollPassword" of 
Appendix B page 29, and other functions called by "enrollPassword" and described 
below. In block 650 the DSP calibrates the subscriber's voice port by playing a pre- 
recorded message such as "please remain silent after the tone." In block 652 the DSP 
executes the "VP_verify" function of Appendix B and its subroutine VP_VCALIB. 
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h. Calibration of Spoken Password 

As is known in the art, before password verification occurs, each voice port 
must be calibrated to filter out background noise. However, in the preferred 
embodiment, port calibration is imperceptible to users. Preferably calibration is carried 
out once on each call before enrollment or verification. Users perceive only a brief- 
pause before enrollment instructions are given. 

The preferred embodiment provides a calibration method which both measures 
and compensates for ambient noise, and build the silence model of HMM. The silence 
15 model is factored into a subscriber's reference template during enrollment. 

Alternatively, the silence model is factored into the test input at verification time. 

Detailed descriptions of hidden Markov models as implemented in the prior art 
can be found in the following publications, the contents of which are hereby 
incorporated by reference: Itakura et al, "Distance measure for speech recognition 
20 based on the smoothed group delay spectrum" , IEEE Conf. on Acoustics, Speech and 

Signal Processing (ICASSP), IEEE, 1987, pp. 1257-1260: Poritz, "Hidden Markov 
models: a guided tour" ICASSP, IEEE, 1988, pp. 7-13; Wilpon et al.. "Automatic 
recognition of keywords in unconstrained speech using HMM's", IEEE Trans, on 
Acoustics, Speech & Signal Processing (ASSP), v 38 n 11, Nov. 90, pp. 1870-1878: 
Bellegarda et al,. "Tied mixture continuous parameter modeling for speech 
recognition". IEEE Trans, on ASSP. v 38 n 12, Dec. 90, pp. 2033-2045: Savic et al.. 
"Variable parameter speaker verification system based on hidden Markov modeling". 
ICASSP 1990, pp. 281-284; Rabiner, "A tutorial on HMM's and selected applications 
30 in speech recognition", Proc. IEEE v 77 n 2, Feb. 89, pp. 257-286; Schwartz et al., 

"Robust smoothing for discrete HMM's", ICASSP 1989, pp. 548-551; Velius, 
"Variants of cepstrum based speaker identity variation", ICASSP 1988, pp. 583-586; 
Murthy, "Speech processing using group delay functions". Signal Processing 3-1-91. 
pp. 259-267; Meyer et al.. "Design and evaluation of optimal cepstral lifters for 
accessing articulator codebooks", IEEE Trans, on Signal Processing v 39 n7 Jul. 91, 
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pp. 1493-1502; Wang Ren-hua et al., "A weighted distance measure based on the fine 
structure of feature space: application to speaker recognition", ICASSP 1990, pp. 273- 
276; Naik et al., "Speaker verification over long distance telephone lines", ICASSP 
1989, pp. 524-527; Murthy, "Formant extraction from phase using weighted group 
delay function". Electronics Letters 1 1-9-89, pp. 1609-161 1 : Wilbur et al., "Consistent 
speaker identification via Wigner smoothing techniques". ICASSP 1988, pp. 591-594; 
Yuan-cheng Zheng et al., "Text-dependent speaker identification using circular 
HMM's, ICASSP 1988, pp. 580- 582; U.S. Patents Nos. 4.989.249 Oka et al.; 
15 4,903,305 Gillick et al,; 4,852.172 Taguchi; 4,829,574 Dewhurst et al.: 4,264.959 

Blass; 4.227,177 Mosjier; and 4.076,960 Buss et al.. 

The invention also provides means for filtering a voice response to separate 
noise and to cope with a varying input which does not exactly match a template. Noise 
can include background noise such as car noise or other speakers in the same room as 
20 the subscriber. Other noise sources include dirty fan lines, open windows . radios, and 

variances in performance of microphones used in telephone sets. 

Preferably calibration is performed in about 720 ms. (about 3/4 second) in 
contrast to prior art methods which require 2880 ms (2.88 seconds) or more. 

As is known in the art. a hidden Markov model has two components: a finite- 
state Markov chain and the output probabilistic distributions which turn the state path 
into a time series. The most frequently used output probabilistic distributions are 
Gaussian or Laplacian functions. In the present invention, a novel spectral distance 
measurement method, based on the smoothed LPC group delay spectrum, gives a stable 
30 recognition performance under variable frequency transfer characteristics and additive 

noise. By using smoothed group delay weighting, the invention modifies the 
logarithmic Laplacian density function. Weighted cepstral coefficients are used to 
measure the maximum likelihood function of the HMM. These methods reduce the 
degradation of recognition performance caused by sensitivity of different microphones. 
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The invention also uses higher order normalized cepstral coefficients and a 
large number of states of the HMM to characterize the acoustic variability of speakers. 
Unlike the prior art, in the method of the invention the order of cepstral coefficients 
is increased from 10th to 20th order, and the number of HMM states is increased from 
10 to 14. 



c. Enrollment 

After calibration the DSP plays a "ready" prompt in block 654 ? such as 
15 "please say YES when you are ready to begin enrollment." In block 656 the DSP 

waits to receive a "YES M response by calling the function "ynRecog" of Appendix B 
page 10. As shown in Appendix B ? the "ynRecog" function uses a speaker independent 
template of the words YES and NO. If either VP_verify or ynRecog fail, the 
enrollPassword function terminates. 

If YES is recognized then the DSP plays a "password" prompt in block 658 ? 
such as "please speak your password after each tone." Control is passed to block 662 
of FIG. 6b in which the DSP receives a plurality of password utterances and creates 
a verification template. Block 662 is preferably the "VFenroli" function of Appendix 
B page 25 ? and the functions called therein. In block 664 the DSP saves the 
verification template preferably using the "saveTemplate" function of Appendix B page 
26. This ends the enroll password mode and control is passed via state 2 to block 640 
of FIG. 6a. 

30 2. Enroll Password - Alternate Embodiment 

a. General 

The spoken password feature of the present invention has three separate modes: 
calibration; enrollment, or initial entry of a password; and verification when a 
subscriber desires to access a voice mailbox. In an alternate embodiment of the 

35 

invention all three modes are integrated with a voice mail application program such as 
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the InfoMail application discussed above. In one preferred embodiment the application 
program operates according to the menu flow shown in FIG. 3 and the logical flow 
shown in FIGs. 4a to 4e, inclusive. A key to the symbols used in FIGs. 4a to 4e 
appears as Appendix D hereto. FIGs. 4a to 4e show general states, prompts, and 
messages of an alternate embodiment. 

Referring to FIG. 3, a voice mail application program preferably includes at 
least three menus, namely a main menu 1300, a personal profile menu 1304. and a 
password options menu 1310. As is known in the art, a menu comprises a verbally 
15 spoken list of options stored by the VMS in a mass storage device such as disk drive 

38. When a subscriber dials the VMS. the VMS answers by first playing back the 
main menu 1300. The main menu is implemented in a recording which says: 

"To review messages, press 1. To scan message info, press one-one. To send 
messages, press 2" ... and so forth for each option. A subscriber selects options by 
20 pressing the corresponding DTMF button on a telephone or by speaking digits. 

The main menu 1300 includes a personal profile change option 1302 preferably 
accessed by pressing 3. which causes the VMS to play back a list of options in the 
personal profile menu 1304. If the subscriber presses the "*" button then the main 
menu is replayed as indicated by arrow 1309. A change password option 1306 is 
selected if the subscriber presses or says "1". As indicated by arrow 1307, the VMS 
next plays back the password options menu 1310, which has an enter numerical 
(DTMF) password option 1312 and an enter spoken password option 1314. If the 
subscriber presses "*" then the personal profile menu-is replayed as indicated by exit 

30 option 1315. 

Logical operation of the enter numerical password option 1312 is shown by 
reference number 1316. The subscriber is expected to dial DTMF digits for the 
password, terminated by a as shown by the DIAL step 13 18. Thereafter the VMS 
plays back the password, as shown by HEAR step 1320. preferably by synthesizing 
speech corresponding to the entered password digits and routing the synthesized speech 
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to the subscriber's telephone handset. As shown by the CHOOSE step 1322. the VMS 
next plays prompts requesting the subscriber to cancel the password by pressing "*" 
or accept the password by pressing If acceptance is chosen (reference number 

1324) then the VMS plays back the password options menu 1310 and the subscriber 
can perform further functions. 

If the enter spoken password option 1314 is selected then at a CHOOSE step 
1326 the VMS plays a message instructing the subscriber to choose an enable spoken 
password option 1328, a disable spoken password option 1330, or a record spoken 
15 password option 1332. Both the enable spoken password option 1328 and disable 

spoken password option 1330 operate like toggle switches to cause the VMS to either 
turn password use on or off. 

b. Enrollment of Spoken Password 
If the record new spoken password option 1332 is selected, then at a SPEAK 
20 s tep 1334 the VMS prompts the subscriber to speak a new password into the telephone. 

This step constitutes the enrollment mode discussed in detail below. After receiving 
a spoken password, the VMS plays back the password in the HEAR step 1336. If the 
password is acceptable to the VMS, as discussed below, then a confirming message is 
played at step 1338 and the password options menu is replayed as shown by arrow 
1340. If the password is rejected by the VMS as shown at step 1342, as discussed 
below, then the enrollment process is repeated as indicated by arrow 1346. 

FIG. 4a shows logical flow of the personal profile menu 1304. State 300 of 
FIG. 4a is entered when a subscriber selects the change personal profile option 1302 
30 from main menu 1300 of FIG. 3. Message 3001 is played back and contains a list of 

options. Message 3001 comprises a plurality of menu messages shown in Tables 

2(a) and 2(b) and varies depending on the VMS context. If the context is 
STANDARD, then the VMS plays back the messages shown in Table 2(a). If the 
context is BRIEF then the Table 2(b) messages are played. 

35 
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After playback of message 3001, state 301 is entered in which the VMS waits 
for the subscriber to press a DTMF digit. In an alternative embodiment the VMS waits 
for the subscriber to speak a digit. States 404 to 414 correspond to options of the 
personal profile menu 1304 shown in FIG. 3. For example, state 414 indicates that 
when a subscriber presses control is passed to state 000 which is the beginning of 
the man menu 1300. States 406 to 422 do not relate to spoken password use and are 
not discussed in detail in this section. 

When the subscriber presses or speaks "1". state 404 is entered which 
15 corresponds to the change numerical or spoken password option 1306 of FIG. 3. A 

three-way branch occurs to state 310, 320 or 302 as shown in FIG. 4a. If the 
subscriber is currently using spoken password capability ("spw = on") then the VMS 
enters state 302, discussed below. 

Otherwise, a DTMF password is produced, and the password options menu 
20 1310 of FIG. 3 is skipped. If the subscriber has the capability of setting his own 

DTMF password, then the VMS enters state 320. Otherwise, the VMS assigns a 
DTMF password using a random number, which is reported to the subscriber in 
message 0033. 

VMS processing in state 302 is shown in FIG. 4b. Message 0797 is played 
which contains a verbal listing of options 1312 to 1315 of the password options menu 
1310. States 424 to 432 correspond to options 1312 to 1315. As shown in FIG. 4b. 
after hearing message 0797, if the subscriber presses then the VMS enters state 
300, i.e. returns to the personal profile menu 1304 described above. If the subscriber 
30 presses "0" or "00" then the VMS returns to state 302 and the password options menu 

is replayed. 

If the subscriber presses "1" (state 424) then DTMF password processing is 
done as described above for option 13 12. If the subscriber presses "2" (state 426) then 
the VMS enters state 324, which initiates processing of a spoken password and is 
shown in detail in FIG. 4c. Subroutine 327 is called to selectively play messages 
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reciting the enable, disable, and record steps 1328, 1330, and 1332 of FIG. 3. 
Subroutine 327 uses the logical flow of FIG. 4e. The VMS tests whether the 
subscriber's spoken password template is properly recorded. If not, then the subscriber 
cannot enable or disable a password, so the VMS follows branch 472 of FIG. 4e and 
plays back only message 01487, which recites the record password option. If a 
template is found, then branch 470 is followed and the VMS then tests whether the 
spoken password is disabled ("mbx spw = OFF"). If so, then message 01485 is 
played which recites the enable password option. Otherwise (spoken password is 

15 enabled) the VMS plays message 01484 which recites the password disable option. 

Then the VMS plays message 01486 to recite the record new password option. 
Thereafter the VMS enters state 474 and returns from subroutine 327. 

Referring again to FIG. 4c, the VMS next enters state 325 to calibrate the 
subscriber's voice port. The VMS then waits for the subscriber to press a DTMF digit 

20 or say a digit and branches to one of states 434 to 444 of FIG. 4c. States 434 to 438 

correspond to steps 1328 to 1332 of FIG. 3. If the subscriber presses "O" (state 440) 
then the VMS returns to state 324 to initiate the spoken password mode again. If the 
subscriber presses "00" (state 442) then the VMS plays warning messages 0798 and 

25 0989 and returns to state 324. If the subscriber presses "*" (state 44 which 

corresponds to step 1344 of FIG. 3), then the VMS enters state 302 and the password 
options menu is replayed. 

If the subscriber presses "5" (state 438) and calibration was successful, then 
the VMS enters state 326 to enroll a password, as described below. If calibration 

30 failed, then a password cannot be recorded properly, so a warning message 01492 is 

played and the VMS enters state 302 described above. 

If the subscriber presses "1" (state 434) to execute the enable spoken password 
option 1328 of FIG. 3, then the VMS tests whether an enrolled password template 
exists. If not, then the VMS plays a warning message and returns to the password 

35 

options menu 1310. If a template exists, then the VMS tests whether spoken password 
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service is currently enabled. If so, then a warning message 01489 is played: otherwise 
a confirming message 01488 is played and the VMS returns to state 302 described 
above. 

If the subscriber presses "2" (state 436) to execute the disable spoken password 
option 1330 of FIG. 3, then the VMS again tests whether an enrolled password 
template exists. If not. then the VMS plays a warning message and returns to the 
password options menu 1310. If a template exists, then the VMS tests whether spoken 
password service is currently disabled. If so, then a warning message 01490 is played: 
15 otherwise a confirming message 01491 is played and the VMS returns to state 302 

described above. 

Spoken password recording is processed by the VMS using the logical flow of 
FIG. 4d. After state 326 is entered the VMS plays back message 0920 followed by a 
synthesized number in state 450 and message 0921 . These three elements result in a 
message such as "Please speak your password eight times," wherein "eight" is the 
synthesized number of state 450. The VMS then calibrates the subscriber's port by 
entering state 327 of FIG. 4d. The VMS is set to record mode in state 452 and waits 
for input from the subscriber in state 454. As input utterances are received they are 

digitally sampled and recorded in the data store . The VMS senses the end of input 

by waiting for a silent interval after a spoken utterance. Then the recorded input is 
tested and the VMS enters either state 456. 458, 460, 462. or 464. If a good recording 
was received (state 456) then the VMS returns to state 327 of FIG. 4d to receive 
another password instance. This process is repeated several times as indicated by the 

30 number in state 450. 

If the user pressed "0" (state 458), then warning messages 0798 and 0989 are 
played and the VMS returns to state 324, i.e. the password options menu. If the user 
pressed "*" to cancel recording a password, then a warning message 0044 is played 
and the VMS returns to state 324. 

35 
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If the recorded password failed one of several integrity tests, or if unintelligible 
DTMF digits were sensed (state 462), then an error is signaled with an error tone in 
state 463. Depending on the type of failed test, a multiway branch is made as shown 
at reference number 466. The VMS attempts to assist the subscriber by playing back 
one of several advisory messages. If the subscriber has failed too many times to record 
a password, then messages 0099 and 01507 are played and the VMS enters state 302, 
i.e. returns to the password options menu. If the recording failed to match a prior 
recording ("bad match"), comprised silence, was too long, or had another type of 
15 error, then the VMS plays back messages 0982, 0989, 0980, or 0982, respectively. 

If the recording comprises DTMF tones for the digits 1 to 9 or "# w , then the subscriber 
is advised that such digits are not permitted as a spoken password, through messages 
01493, 0798, and 0989. After each error condition the VMS enters state 324 to begin 
spoken password entry again. 

20 

c. Verification of Spoken Password 
Referring to FIGs. 5a to 5d, verification forms one state in a process used by 
a subscriber to gain access to the subscriber's voice mailbox on the VMS. The process 
25 of FIGs. 5a and 5d is preferably implemented in the C language program of Appendix 

B which is described in detail below. 

As shown in FIG. 5a, verification occurs after a subscriber initially dials the 
VMS in state 500. In state 500 the VMS performs system functions well known in the 
art, such as initializing a voice port, assigning the port to one of the DSPs on a LIC, 
30 and playing back an initial greeting. In state 700 the assigned DSP waits for the 

subscriber to enter a mailbox identification number ("mbx id"), which preferably is a 
string of DTMF digits. After the id is received, the DSP tests whether the id is known 
by the VMS, preferably by comparing the id to a look-up table stored in the data store 
or on the disk drive 38. If the id is unknown, then control is passed to state 710 which 
is shown in detail in FIG. 5c and described below. 
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If the id is known then the DSP tests whether spoken password service is 
currently enabled for the identified mailbox ("mbx spw=on"), and also whether the 
subscriber has attempted to enter a spoken password fewer than two times (" <2 spw 
tries already")- If these tests are affirmative then control passes to state 702, shown 
in detail in FIG. 5b and described below. 

If spoken password service is not currently enabled, then message 0100 is 
played to prompt the subscriber to enter a DTMF password. In state 701 of FIG. 5a 
the DSP waits for DTMF digits to be pressed or for a timeout to occur. The input 
15 received is tested in state 501, and the DSP then branches to one of four states 502. 

504. 506, or 508. If the subscriber pressed one too many digits ("Max+ 1 "), then state 
502 is reached. The DSP generates an error tone in state 5 10 and plays an explanatory 
message 9000, such as "too many digits entered." Then control passes back to state 
701 to await another entry. 

If the subscriber presses "#". then state 504 is reached. This process assumes 
that the subscriber also entered a series of digits before the "#" entry. Thus, in state 
504. the DSP tests whether a proper set of digits were entered. If not. then control is 
passed on branch 520. On this branch a FIRST flag is set to indicate that the 
subscriber has made an erroneous entry for the first time: then control passes to state 
720. If proper digits were entered, then the DSP compares the entered digits to the 
subscriber's DTMF password stored in the data store 74 or on the disk drives 38. If 
no match is made, then control passes on branch 512 to state 710 in which the 
subscriber can enter a mailbox number. If a match is made, then on branch 514 the 
30 FIRST flag is reset and the DSP tests whether a "Tutorial" flag is set on to indicate 

that the subscriber is unfamiliar with the VMS and desires tutorial information. If the 
Tutorial flag is ON then control passes on branch 516 to state 760 in which tutorial 
messages are played back. Otherwise control passes on branch 518 to state 750 in 
which the subscriber can perform any desired voice mailbox function. State 750 is not 
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described in detail herein but it preferably includes the functions described in the 
above-noted reference manuals of Digital Sound Corporation. 

State 506 is reached when the subscriber presses M *" on the telephone keypad 
to cancel entry of a DTMF password. In state 506 the DSP tests whether any digits 
were entered before the "* M and also tests how many cancellations have been requested 
before. If digits were received then control passes to branch 522. If too many prior 
cancellations have been made, e.g. more than 19 cancellations, then control passes on 
branch 526 to state 528 in which an error tone is played. The DSP then plays warning 
15 messages 0098, 0292, and 0139, and then branches state 1000 in which the subscriber 

is disconnected from the VMS. If fewer than 19 cancellations have occurred then a 
warning message 0351 is played and control is passed back to state 700 to enable the 
subscriber to try again. 

If no digits were received before the "*", then control is usually passed to state 

20 

710 to receive entry of a mailbox number. If too many prior cancellations have been 
made, e.g. more than 19 cancellations, then control passes on branch 530 to state 531 
in which an error tone is played. The DSP then plays warning messages 0098, 0292, 
and 0139, and then branches state 1000 in which the subscriber is disconnected from 

25 the VMS. 

State 508 is reached when a timeout occurs, i.e. no response is sensed in state 
701 for a predetermined period of time such as five seconds. In state 508 the DSP 
tests whether any DTMF digits were received before the timeout and also whether two 
previous timeouts have occurred. If two previous timeouts have occurred then* control 

30 i s passed on branch 536 in which the DSP plays warning messages 0292 and 0139. 

Then the subscriber is disconnected in state 1000. If no digits were received then 
control passes on branch 534 back to state 700, causing the DSP to prompt the 
subscriber to enter a password again. If one or more digits were received before the 
timeout then control passes on branch 532 in which the DSP plays warning message 

35 

0178 and then returns to state 701 to wait for entry of a new DTMF password. 
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Referring now to FIG. 5b. spoken password entry is processed starting at state 
702 which is reached when, as noted in FIG. 5a, the subscriber's mailbox id is 
recognized, spoken password service is set on. and fewer than two attempts to enter 
a spoken password have been made. In state 702 the DSP attempts to calibrate the 
subscriber's voice port as described above. If the DSP is unable to calibrate properly 
then control passes on branch 540 in which the DSP plays back a warning message 
0100. The DSP then passes control to state 701 to require the subscriber to enter a 
DTMF password or exit the system. 
15 Branch 542 is reached if calibration is successful. The DSP plays a prompt 

message 0795 and then generates a "ready" tone in state 544. In state 546 the DSP 
waits tor a voice utterance or a DTMF digit to be pressed, and tests whether voice or 
DTMF digits were received. If DTMF digits were received, state 550, then control 
is passed to state 701 which processes a DTMF password as described above. Thus. 
20 this process enables the user to enter either a spoken password or a DTMF password 

with proper automatic processing and verification of either type of password. In 
particular, the process properly handles a DTMF password entered when the process 
is "expecting" a voice password. 

If no DTMF digits are detected then state 548 is entered to process a voice 
utterance by the subscriber. As described in detail below, the DSP generates a hidden 
Markov model template for the utterance and compares the input template to the 
subscriber's stored template which was generated in the enrollment mode described 
above. In one preferred embodiment the DSP is also provided with means', in a 
30 subroutine of a stored program, to provide a subscriber with access to a mailbox 

extension. If extension service is enabled then control is passed on branch 552 to state 
720 (not shown in detail) to process access to a mailbox extension. 

If a template match fails in state 548 then control is passed on branch 554 to 
state 710. in which the DSP processes entry of a mailbox id number. If a match is 
made then control passes on branch 556 in which the DSP tests whether the subscriber 
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is a new user, i.e. whether Tutorial is set ON. If so, then control passes on branch 
558 to state 760 in which tutorial messages are played. Otherwise control passes on 
branch 560 to state 750 in which the subscriber can perform any desired voice mail 
function. 

Referring now to FIG. 5c. state 710 is reached to process subscriber input of 
a mailbox id. Message 0127 is played to prompt the subscriber to enter DTMF digits 
for a mailbox id. In state 71 1 the DSP waits for DTMF digits to be pressed or for a 
timeout to occur. The input received is tested in state 570, and the DSP then branches 
15 to one of four states 572, 574, 576, or 578. If the subscriber pressed one too many 

digits ("Max + 1 "), then state 572 is reached. The DSP generates an error tone in state 
580 and plays explanatory messages 0214 and 0268 such as "too many digits entered." 
Then control passes back to state 701 to await another entry. 

If the subscriber presses then state 574 is reached. This process assumes 
20 that the subscriber also entered a series of digits before the "#" entry. Thus, in state 

574, the DSP tests whether a proper set of digits were entered. If not, then control is 
passed on branch 582. On this branch a FIRST flag is tested to determine that the 
subscriber has made an erroneous entry for the first time. If so. then message 0100 
25 is played and control passes to state 701. If proper digits were entered, then the DSP 

compares the entered digits to the subscriber's DTMF mailbox id stored in the data 
store 74 or on the disk drives 38. If no match is made, then control passes on branch 
584. On this branch the DSP tests whether mailbox extension service is enabled and 
if so, branch 588 is taken to state 720. If extension service is disabled then the DSP 
30 tests whether three failed matches have occurred. If so, then branch 592 is taken to 

state 602 in which the DSP plays an error tone followed by warning messages 01 16 
and 0139. Then the subscriber is disconnected in state 1000. If fewer than three failed 
matches have occurred then branch 590 is taken to state 594, in which the DSP 
generates an error tone and then plays warning message 0214. Next the DSP tests 

35 

whether spoken password service is enabled and if so, branch 596 is taken, in which 
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the DSP prompts the subscriber to enter a spoken password, which is then processed 
via state 702. 

As described above with reference to FIG. 5b, branch 596 of state 710 can be 
reached on branch 554 after a failed spoken password match test followed by a failed 
mailbox id entry. Thus, this process is structured to require a subscriber to 
successfully enter both a spoken password and a mailbox id, and to permit successful 
entry of one even if entry of the other initially fails. 

If mailbox id match is made, then on branch 586 the DSP tests whether a 
"Tutorial" flag is set on to indicate that the subscriber is unfamiliar with the VMS and 
desires tutorial information. If the Tutorial flag is ON then control passes on branch 
598 to state 760 in which tutorial messages are played back. Otherwise control passes 
on branch 600 to state 750 in which the subscriber can perform any desired voice 
mailbox function. State 750 is not described in detail herein but it preferably includes 
20 the functions described in the above-noted reference manuals of Digital Sound 

Corporation. 

State 576 is reached when the subscriber presses "*" on the telephone keypad 
to cancel entry of a DTMF password. In state 576 the DSP tests whether any digits 

25 were entered before the " *" and also tests how many cancellations have been requested 

before. If digits were received then control passes to branch 608. If too many prior 
cancellations have been made, e.g. more than 19 cancellations, then control passes on 
branch 604 to state 606 in which an error tone is played. The DSP then plays warning 
messages 0098, 0292, and 0139. and then branches state 1000 in which the subscriber 

30 is disconnected from the VMS. If fewer than 19 cancellations have occurred then a 

warning message 0350 is played and control is passed back to state 710 to enable the 

subscriber to try again. 

If no digits were received before the " *" , then control is usually passed to state 
700 to receive entry of a spoken password. If too many prior cancellations have been 
made, e.g. more than 19 cancellations, then control passes on branch 612 to state 614 
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in which an error tone is played. The DSP then plays warning messages 0098. 0292, 
and 0139, and then branches state 1000 in which the subscriber is disconnected from 
the VMS. 

State 578 is reached when a timeout occurs, i.e. no response is sensed in state 
701 for a predetermined period of time such as five seconds. In state 578 the DSP 
tests whether any DTMF digits were received before the timeout and also whether two 
previous timeouts have occurred. If two previous timeouts have occurred then control 
is passed on branch 620 in which the DSP plays warning messages 0292 and 0139. 
15 Then the subscriber is disconnected in state 1000. If no digits were received then 

control passes on branch 618 back to state 710, causing the DSP to prompt the 
subscriber to enter a mailbox id again. If one or more digits were received before the 
timeout then control passes on branch 616 in which the DSP plays warning message 
0178 and then returns to state 711 to wait for entry of a new mailbox id. 
20 This concludes description of the alternate embodiment of spoken password 

enrollment. 

4 Enroll Recipient's Name and Word STOP Mode (Preferred Embodiment) 
Referring again to FIG. 6a, if the test of block 640 is true then the enroll 
recipient's name and word mode begins with calibration at block 670. Blocks 670 to 
676 correspond directly to blocks 650 to 656 of the enroll password mode and 
preferably use the same functions of Appendix B as the enroll password mode. After 
calibration and receipt of a YES/ready response, in block 678 the DSP plays a "stop" 
30 prompt such as "please say STOP after each tone." In block 680 the word STOP is 

enrolled for the subscriber preferably using the "RCenroll" function of Appendix B 
page 27. This function uses speaker dependent recognition methods to create a 
digitally stored template of the subscriber speaking the word STOP. The template can 
be stored by copying it to a word table such as the "wordTab" structure of Appendix 
B. 
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The loop of blocks 684 to 694 enables a subscriber to enroll a plurality of 
recipients' names. The number of names allowed is defined in Appendix B in a 
constant "NUM_NAMES" which preferably is five. Of course, NUM_NAMES could 
be larger depending on the amount of available memory in the data store. In block 684 
the DSP tests whether the number of currently enrolled names ("i") is less than 
NUM_NAMES. If so, then in block 688 the DSP plays a speak name prompt such as 
"after each tone, please speak the name of a message recipient." Then the name 
spoken by the subscriber is enrolled preferably using the RC_enroll function of 
15 Appendix B. and copied to the word table in blocks 690, 692. The value of "i" is 

accumulated by 1 in block 694 and control returns to block 684. 

After enrollment of the maximum number of names control is passed to block 
686 in which the DSP saves the enrolled template in the data store and on the disk 
drive 38, preferably using the "closeFiles" function of Appendix B. Via state 4. 
20 control is passed to block 642 of FIG. 6a. 

S Kftview New Message Mode with P assword Verification 
The review new message mode 636 is shown in detail in FIGs. 7a to 7e. 
Before the DSP plays back stored messages, the subscriber's password is verified via 
blocks 700 to 712. In block 700 the DSP loads the subscriber's spoken password 
template by calling the "loadTempIateV" function of Appendix B page 8. The 
subscriber's voice port is calibrated in block 702 by calling the "CalibrateV" function 
of Appendix B page 9. In block 704 the DSP plays, a "speakpw" prompt such as 
30 "please speak your password after the tone." The subscriber's spoken password is 

received and recorded by the DSP in block 706 using the "VP.verify" and 
"VP VSTART" functions. Then the DSP tests whether the password is acceptable in 
block 708 by testing variable parameters returned by the VP_verify function. If the 
password is rejected, control passes to block 710 in which the DSP plays a "reject- 
message such as "your password has been rejected ... goodbye." The verify password 
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mode then ends at block 712 in which the DSP disconnects the subscriber and closes 
its voice port. 

If the password is accepted then in block 714 the DSP loads a vocabulary table, 
in the form of a disk file stored on the disk drive 38, into memory using the 
"loadVocab" function of Appendix B page 7. In block 716 the DSP plays a "number" 
prompt advising the subscriber of how many messages are currently in its mailbox, 
such as "you have two new messages and one saved message." In block 7 18 of FIG. 
7b, the DSP next plays a review prompt such as "do you want to review your messages 
15 ... please say YES or NO after the tone." In block 720 the DSP waits for a YES or 

NO response using the "ynRecog" function of Appendix B. During block 722 the DSP 
tests whether an excessive delay (timeout) has occurred. If so, then in block 724 the 
DSP plays a "timeout" message such as "you are now exiting voice mail ... goodbye." 
In block 726 the DSP then disconnects the subscriber. If no timeout occurred, then in 
block 724 the DSP tests whether the utterance received by "ynRecog" was a word, 
rather than non-speech or a DTMF digit, by the statement [ event- 
> dass = = VP EWORD ]. If this test is false then the subscriber said something other 
than a word. Therefore, control passes to block 726 in which the DSP plays a repeat 
prompt such as "your voice has not been recognized ... please say YES or NO again." 
Control is then passed back to block 720. 

If YES or NO was said, two tests are done in blocks 728 and 730 to determine 
which word was spoken. In block 728, using the statement " event- > value = =0" the 
DSP tests whether the utterance was YES. If not, then in block 730 the DSP tests for 
30 NO. If the test of block 730 is false then in block 732 the DSP plays the "repeat" 

message used in block 726. 

If YES was sensed then control passes to a subroutine 734 called "messagel" 
in Appendix B. As shown in block 736, the DSP plays back three consecutive messages 
comprising a message header, a person's name, and the time the message was sent. 
In the embodiment of Appendix B these messages are given fixed values, namely "first 
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new message from", "Louise Jennings." and "sent at 9:15 a.m." Of course, these 
messages can be loaded with any desired value by the DSP at the time a subscriber 
records a message on the VMS . To play back the actual contents of the first message, 
the DSP then calls a subroutine "playRecog" having the logic of blocks 738 to 748 of 
FIG. 7c. During this subroutine a message is played back and during playback, the 
DSP senses whether the listening subscriber says STOP. If so then message playback 
is interrupted and the next message is played. In the embodiment of Appendix B. these 
logical operations begin in block 738 when the DSP loads a pre-recorded speaker 
dependent template of the word STOP using the function "loadTemplate" of Appendix 
B page 7. This template is recorded, as described above, during enrollment of the 
word & STOP. In block 740 the DSP calibrates the listening subscriber's voice port by 
calling the function "calibrateR" of Appendix B page 10. Then the DSP enables 
recognition of the word STOP by calling the functions "VPjrcog" and "VP_START" 
20 in block 742. At this point if the listening subscriber says STOP then the DSP will be 

able to respond by interrupting message playback. Next the DSP plays back the 
message in block 744 having the contents of the "messagel.v" file given above. In 
parallel the DSP waits for an event, such as an utterance of the word STOP, by calling 
the function "getEv" in block 746. After playback of the message is complete, in 
block 748 the DSP disables STOP recognition by calling the functions "VP_vcog" and 
"VP STOP." This block 748 ends subroutine playRecog and processing continues in 
subroutine messagel at state 9 of FIG. 7d. 

Referring now to FIG. 7d, in block 750 the DSP tests whether a word was 
uttered by the listening subscriber using the test "event- > class = = VP_EWORD" . If 
no word was uttered then the message ended normally, so control is passed to block 
752 in which the DSP plays an "endmess" message such as "end of message." If the 
test of block 750 is true then in block 754 the DSP tests whether the utterance was the 
word STOP with the test " event- > value ==0". If true then in block 758 the DSP 
plays a "messtop" message such as "message stopped." 
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6. Delete/Erase Message Mode 

Next the subscriber is given the opportunity to erase the message which was 
just played. Preferably this logic occurs through a subroutine "ynRecog" of Appendix 

B page 10 and the logic of blocks 764 to of FIGs. 7d and 7e. In block 760 the 

DSP loads a speaker independent template for the words YES and NO using the 
function "loadTemplate" of Appendix B. Then in block 762 the subscriber's port is 
calibrated using the function "calibrateR". Next in block 764 the DSP plays an erase 
prompt such as "Do you want to erase this message ... please say YES or NO after the 
tone." 

As shown in FIG. 7e, in block 766 the DSP next enables recognition of YES 
or NO by calling the function "VP_rcog" of App. B. The DSP waits for an utterance 
using the function "getEv" of App. B in block 768. After an receiving an event, which 
can be an utterance, a DTMF digit, or a timeout, in block 770 the DSP disables 
YES/NO recognition using the "VP_rcog" and "VP_STOP" functions. In block 722 
the DSP tests whether a timeout occurred. If so, then in block 774 the DSP plays the 
timeout message used above in enrollment mode, and returns control to messagel in 
block 776. 

If no timeout occurred then the DSP next tests whether the subscriber said 
YES, NO, or some other utterance. In block 778 the DSP tests whether the subscriber 
spoke a word. If not, then in block 780 the DSP plays a repeat message to prompt the 
subscriber to repeat the utterance. Next the DSP tests whether the utterance is not 
recognizable as YES or NO, i.e. it is a "bad utterance." If so then the repeat message 
is played in block 780. Thus the test of block 784 is reached only when YES or NO 
has been uttered. If YES was uttered, i.e. the test of block 784 is true, then the DSP 
plays a confirming message such as "message erased." If NO was uttered, i.e. the test 
of block 784 is false and the test of block 788 is true, then the DSP plays a confirming 
message such as "message saved." 
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Processing of the subscriber's first new message is thus complete. If further 
new messages exist in a subscriber's message queue, then as indicated by block 791. 
the DSP plays the next new message and repeats blocks 736 to 790 as described above. 
In such a case the subroutine "messagel" would play back the next message in the 
queue rather than the specific message content described above. 



7, Review Saved Messages Mode 

The subscriber is next allowed to review previously saved messages using a 
15 review saved message routine 800 of FIG. 8a. As shown in FIG . 7b, the review saved 

message routine can also be reached from block 730 of FIG. 7b. In block 802 of FIG. 
8a T the DSP plays a review prompt such as "do you want to review your saved 
messages ... please say YES or NO after the tone." In block 804 the DSP waits for 
a YES or NO response using the "ynRecog" function of App. B. During block 806 
20 m e DSP tests whether timeout has occurred. If so. then in block 808 the DSP plays 

a "timeout" message such as "you are now exiting voice mail ... goodbye." In block 
810 the DSP then disconnects the subscriber. If no timeout occurred, then in block 
812 the DSP tests whether the utterance received by "ynRecog" was a word, rather 
than non-speech or a DTMF digit, by the statement [ event- > class = =VP_EWORD 
]. If this test is false then the subscriber said something other than a word. Therefore, 
control passes to block 814 in which the DSP plays a repeat prompt such as "your 
voice has not been recognized ... please say YES or NO again." Control is then 
passed back to block 804. 
30 If YES or NO was said, two tests are done in blocks 816 and 822 to determine 

which word was spoken. In block 816. the DSP tests whether the utterance was YES. 
If not. then in block 822 the DSP tests for NO. If the test of block 822 is false then 
in block 824 the DSP plays the "repeat" message. If YES was sensed then in block 
818 the saved message is played. In the embodiment of App. B this is done by a 
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subroutine called M message3". Thus the processing of FIG. 8a blocks 802 to 818 is 
similar to blocks 718 to 736 of FIG. 7b. 

8. Delete/Erase Saved Message Mode 

The subscriber is next given the opportunity to erase a saved message. This 

is done in the logic of blocks 820 to 840 of FIGs. 8a and 8b. The logic of these blocks 

is not described in detail herein since these blocks correspond to blocks 764 to 790 of 

FIGs. 7d and 7e. as shown in Table 3. 

15 Table 3 — Erase Message Logic 

FIG. 7d/7e corresponds to FIG. 8a/8b and to FIG. 8c 

764 820 851 

766. 768, 770 822 852 

772 824 854 

774 826 856 

776 828 858 

778 830 860 

780 832 862 
782 

784 834 864 

786 836 

788 838 866 

790 840 870 

842 868 

However, as shown in FIG. 8b. in block 842 the subscriber is prompted to repeat an 

utterance if neither YES nor NO were recognized. This ensures that previously saved 

messages are erased whenever possible. In the embodiment of App. B processing of 

saved messages ends at state 13 of FIG. 8b. In an alternate embodiment the VMS can 

provide a queue of saved messages on the disk drive 38, and provide means for 

processing more messages in the queue at state 13. For example, state 13 could be 

replaced with subroutine return logic to cause the DSP to repeat blocks 802 to 842 for 

each saved message in the queue. 

35 



20 



25 



30 



9. Send Message Mode 



WO 93/13518 



PCT/US92/11285 



1 



-42- 



10 



The subscriber is next given a chance to send a message to another subscriber, 
called the "receiving subscriber", in a send message routine 850 of FTGs. 8b and 8c. 
In block 851 the DSP plays a send message prompt such as "do you want to send a 
message ... please say YES or NO after the tone." In blocks 852 to 872 the DSP then 
processes a YES or NO utterance. The logic of blocks 852 to 872 corresponds 
generally to blocks 804 to 824 of FIG. 8a, as shown above in Table 3. 

If a YES is uttered, i.e. the test of block 864 is true, then the DSP enters a 
subroutine 880 called "nameRecog" in App. B using the logic of FIG. 8c and 8d. In 
15 b i 0C k 882 the DSP loads speaker dependent templates of message recipients' names. 

using the "loadTemplate" function of App. B. These templates are pre-recorded by a 
subscriber in the name enrollment mode described above. Then the DSP calibrates the 
sending subscriber's voice port by calling the "calibrateR" function of App. B. The 
DSP next plays a "recipname" prompt to the sending subcriber, such as "pleas% say 
20 the recipient's name after the tone." In block 888 the DSP enables speaker dependent 

recognition by calling the "VP_rcog" and VP_START functions. Then the DSP waits 
for an event, such as a spoken utterance or timeout, in block 890 by calling "getEv". 
When an event occurs the DSP disables recognition in block 892 by calling "VP_rcog" 
and VP_STOP. In block 894 the DSP tests whether the event was a word. If not. then 
in block 896 the DSP plays a "namerejecf prompt such as "the recipient's name has 
not been recognized." Then control is transferred via state 13 to block 851 to enable 
the subcriber to try again. If the test of block 894 is true then in block 898 the DSP 
plays the recipient's name. In the embodiment of App. B this is done by playing the 
30 content of a "nameOO" voice file. In an alternate embodiment, the test of block 894 

is supplemented with an additional test to match the sending subscriber's utterance to 
one of several digitally stored templates for different recipients' names. This matching 
can be carried out in a manner similar to the speaker dependent spoken password 
verification methods described below. Then the matched name is played back in block 
898. 



25 



35 



WO 93/13518 



PCT/US92/11285 



1 



-43- 

5 

The sending subscriber then is allowed to record a message to be sent in blocks 
900 to 906. In block 900 the DSP plays a record message prompt such as "please 
record your message after the tone." In block 902 the DSP records the voice of the 
sending subscriber in a message file on the disk drive 38 by calling the function 
"VCjecord." This function is one of a plurality of voice primitive functions in a file 
"VC.h" commercially available from Digital Sound Corp. Appendix E hereto shows 
the contents of the functions of the VC.h file and the files VS.h and VP.h referenced 
in Appendix B. Any known method of digitally recording a subscriber's voice can be 

15 US ed in block 902. In block 904 the DSP prompts the sending subscriber with a review 

prompt such as "Do you wish to review your message ... please say YES or NO after 
the tone." In block 906 the DSP waits for an utterance by the sending subscriber and 
tests whether the utterance is YES or NO. 

The YES/NO recognition processing of blocks 906 to 918 is implemented using 

20 logical steps similar to those described above for other YES/NO processing. As shown 

in FIG. 9b, if a NO utterance is detected, then in block 922 the DSP plays a 
confirming message such as "message saved" to advise the sending subscriber that the 
message has been recorded and saved. Using processing steps known in the art, the 

25 saved message is stored in association with means for recipient identification to enable 

the receiving subscriber to obtain the message. For example, the saved message can 
be tagged with the recipient's mailbox id or added to the recipient's message queue. 
If a YES utterance is detected in block 916, then in block 926 the DSP plays back the 
recorded message. 

30 Referring again to FIG. 9b. the sending subscriber is then allowed to choose 

whether or not to send the message in the logical steps starting at block 928. In block 
928 the DSP plays a send message prompt such as "Do you wish to send your message 
... please say YES or NO after the tone." In steps 930 to 946 the DSP tests whether 
a YES or NO is uttered using logic similar to the YES/No processing described above. 

35 

If a YES is detected then in block 944 the DSP plays a confirming message such as 
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"message sent. " If a NO is detected then the subscriber desires to discard the message, 
so in block 948 the DSP plays a confirming message such as "message erased." If the 
utterance cannot be identified then in block 942 the DSP plays a repeat prompt such 
as "response not recognized ... please say yes or no again." 

Thus, the above logic enables voice command of VMS features including 
message playback, message sending, message saving, deletion, and reviewing. The 
invention also enables a subscriber to control message playback by saying STOP during 
playback. 



1 P. Other Features and Processing Modes 

In the preferred embodiment, the following speaker verification parameters are 
configurable: sensitivity threshold, number of enrollment iterations, andmaximum 
silence duration. Preferably the parameters are implemented using constants in the 
program of App. B. Configurability is done by changing the constants and re- 
compiling the program. 

In the preferred embodiment the use of a password can be enabled and 
disabled, i.e. turned on and off. by a subscriber using DTMF commands. When 
spoken password service is turned off. the template is preferably retained by storage 
on a mass storage device such as disk drive 38. 

Preferably the VMS includes means for universal password entry, by which a 
subscriber can enter either a spoken password or a DTMF keypress password, without 
first telling the VMS which type of password will be used. Also in the preferred 
30 embodiment, the spoken password feature employs an ordinary telephone microphone 

and operates in real time. Preferably the amount of memory storage tor a template is 
low, allowing the system to store templates for a large number of subscribers. In one 
preferred embodiment a template can be stored in about 1 100 bytes per subscriber per 
template. 
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In an alternative embodiment, the VMS can further comprise an adaptive 
template capability whereby a subscriber's template is modified to incorporate new 
information each time the subscriber accomplishes mailbox login after password 
verification. One possible application is when the subscriber has laryngitis, a head cold 
or other impairment which renders the subscriber unable to properly match the 
template. An adaptive template capability also precludes the need for the subscriber 
to change the template when line conditions change. This can occur when the 
subscriber acquires a new telephone set or moves to a different office. Adaptive 
15 template capability also causes the password template to become more accurate as the 

subscriber uses it more often. 

In another alternative embodiment, the VMS comprises means for multiple 
template entry, enabling a subscriber to record a template from each telephone he 
expects to use to gain access to his mailbox. For example, a subscriber could record 
20 templates from his office phone, car phone, and home phone, each of which would 

have different characteristics. The VMS would select the correct template using the 
calling party id. Moreover, in an alternative embodiment, the VMS can comprise 
means for storing a plurality of confidence levels, each confidence level being 
25 associated with a different template. This would improve performance at each different 

phone location. 

Preferably the spoken password is both recorded digitally and encoded using 
a secure encoding template. The encoding template, discussed in detail below, uses a 
combination of linear predictive coding (LPC) and cepstral coefficients. Using this 
30 template, the user password cannot be decoded if it is intercepted by an unauthorized 

listener. 

Thus, the invention provides numerous advantages over the prior art. A 
spoken password is superior to a DTMF password because use of a DTMF password 
always enables imposters to access subscriber mailboxes. Spoken password access is 
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easier and usually faster than entry of a DTMF password, and allows hands free 
control of a VMS. 

The invention may be practiced in many ways other than as specifically 
described herein. For example, different quantities of taps can be used in the buffering 
functions depending on the desired degree of spike resistance. In an alternative 
embodiment, the VMS can include means for secure login in which both a DTMF 
password and a spoken password must be entered to gain access to a subscriber 
mailbox. In yet another alternative embodiment, the VMS can include means for 
configurable response, by which a subscriber can specify whether a password should 
comprise spoken password only, DTMF password only, spoken OR DTMF. and no 
password. 

Thus, the invention should be given the full scope of the appended claims, in 

which: 
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WHAT IS CLAIMED IS: 

1. A voice controlled digital messaging system comprising: 
at least one subscriber telephone line coupled to a line interface controller, said 
controller for digitizing voice signals received on the telephone line; 

a storage device storing a plurality of digitized voice prompts; 
signal processor coupled to the line interface controller and to the storage 
device for controlling storage, retrieval, playback, and sending of voice messages, 
comprising 

15 means for controlling access to the system by verification of a spoken 

password; 

means for controlling playback of voice messages stored on the 
storage device, comprising means for interrupting playback when a system 
subscriber inputs a voice command during playback; and 

means for controlling storage, retrieval, and sending of voice 
messages in response to voice commands of YES and NO given by the 
subscriber. 
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25 



2. The system of claim 1 wherein the means for controlling access to the 

system further comprises means for enrollment of a password template, the template 
comprising a digital representation of features of a spoken password signal provided 
by the subscriber. 

30 3. The system of claim 2 wherein the means for enrollment further 

comprises means for receiving a first plurality of input spoken password utterances 
from the subscriber, means for creating the template based on a second plurality of the 
utterances fewer in number than the first plurality, and means for testing the template 
by comparing a third plurality of the utterances to the template. 
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4. The system of claim 1 ? wherein the means for controlling playback 

of voice messages stored on the storage device, comprises means for enrolling the word 
STOP spoken by a subscriber. 



5. The system of claim 4 wherein the means for enrolling the word 
STOP further comprises means for receiving a first plurality of input spoken password 
utterances from the subscriber, means for creating the template based on a second 
plurality of the utterances fewer in number than the first plurality, and means for 

15 testing the template by comparing a third plurality of the utterances to the template. 

6. The system of claim 4 wherein the means for controlling storage, 
retrieval, and sending of voice messages further comprises means for speaker 
independent recognition of utterances of YES and NO. 

20 

7. The system of claim 4 wherein the means for controlling storage, 
retrieval, and sending of voice messages further comprises: 

means for prompting a subscriber to select a play message mode for playing 
messages stored on the storage device: 

means for speaker independent recognition of utterances of YES and NO by 

the subscriber; and 

means responsive to a YES utterance for playing back a stored message. 

30 8. The system of claim 4 wherein the means for controlling storage, 

retrieval, and sending of voice messages further comprises: 

means for prompting a subscriber to select a save message mode for saving 
messages recently stored on the storage device: 

means for recognition of utterances of YES and NO by the subscriber: and 
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means responsive to a YES utterance for saving a recently received stored 
message on the storage device. 

9. • The system of claim 4 wherein the means for controlling storage, 
retrieval, and sending of voice messages further comprises: 

means for prompting a subscriber to select an erase message mode for erasing 
messages recently stored on the storage device; 

means for speaker independent recognition of utterances of YES and NO by 

15 the subscriber; and 

means responsive to a YES utterance for erasing a recently received message 

from the storage device. 

10. The system of claim 4 wherein the means for controlling storage, 

20 

retrieval, and sending of voice messages further comprises: 

means for prompting a subscriber to select a review saved message mode for 
replaying messages previously stored on the storage device by the subscriber; 

means for speaker independent recognition of utterances of YES and NO by 

2 5 the subscriber; and 

means responsive to a YES utterance for replaying a message stored on the 

storage device. 

11. The system of claim 4 wherein the means for controlling storage, 
30 retrieval, and sending of voice messages further comprises: 

means for prompting a subscriber to select an erase saved message mode for 
erasing messages stored on the storage device by the subscriber in the save message 
mode; means for speaker independent recognition of utterances of YES and NO by the 
subscriber; and 
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means responsive to a YES utterance for erasing a saved message from the 
storage device. 

12. The system of claim 1 wherein the means for controlling storage, 
retrieval, and sending of voice messages further comprises: 

means for prompting a subscriber to select a send message mode for sending 
messages from the subscriber to one of a plurality of message recipients: 

means for speaker independent recognition of utterances of YES and NO by 

15 the subscriber; 

means responsive to a YES utterance for sending messages from the subscriber 
to one of the plurality of message recipients, comprising: 

means for speaker dependent recognition of the name of a message 

recipient; 

20 means for recording a digitized voice message to the recipient on the 

storage device; 

means for selectively replaying the voice message upon speaker 
independent recognition of a YES utterance from the subscriber. 

25 

13. The system of claim 12 wherein the means for sending messages 
further comprises means for enrolling a plurality of speaker dependent templates 
representing digitized utterances of a plurality of names of message recipients. 

30 14. The system of claims 2 or 13 wherein the template is created using 

a hidden Markov model having at least fourteen states and having cepstral coefficients 
of at least twentieth order. 



35 



15. The system of claim 14 wherein the means for testing the template 
comprises measuring spectral distance using smoothed Laplacian group delay spectrum. 
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16. The system of claim 14 wherein the means for testing the template 
comprises means for weighting the cepstral coefficients using smoothed group delay. 

17. In a data processing system having at least one subscriber telephone line 
coupled to a line interface controller for digitizing voice signals received on the 
telephone line, a storage device having a plurality of digitized voice prompts recorded 
thereon, and signal processor coupled to the memory and to the line interface controller 
and to the storage device; a method for controlling storage, retrieval, playback, 

15 and sending of voice messages, comprising the steps of: 

controlling access to the system by verification of a spoken password; 

controlling playback of voice messages stored on the storage device, 
comprising means for interrupting playback when a system subscriber speaks 
a voice command during playback; and 

controlling storage, retrieval, and sending of voice messages in 
response to voice commands of YES and NO given by the subscriber. 
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18. The method of claim 17 wherein the step of controlling access to the 
method further comprises steps for enrollment of a speaker dependent password 
template, the template comprising a digital representation of features of a spoken 
password signal provided by the subscriber. 

30 19. The method of claim 18 wherein the step of enrollment further 

comprises the steps of: 

receiving a first plurality of input spoken password utterances from the 

subscriber, 

creating the template based on a second plurality of the utterances fewer in 
number than the first plurality, and 
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testing the template by comparing a third plurality of the utterances to the 
template. 

20. The method of claim 17, wherein the step of controlling playback of 
10 voice messages stored on the storage device, includes the substep of enrolling the word 

STOP spoken by a subscriber. 

21 . The method of claim 20 wherein the step of enrolling the word STOP 

15 further steps of: 

receiving a first plurality of input spoken password utterances from the 

subscriber, 

creating the template based on a second plurality of the utterances fewer in 
number than the first plurality, and 
20 testing the template by comparing a third plurality of the utterances to the 

template. 

22. The method of claim 20 wherein the step of controlling storage, 
retrieval, and sending of voice messages further the step of speaker independent 
recognition of utterances of YES and NO. 



25 



23. The method of claim 20 wherein the step of controlling storage, 
retrieval, and sending of voice messages further comprises the steps of: 
30 prompting a subscriber to select a play message mode for playing messages 

stored on the storage device: 

speaker independent recognition of utterances of YES and NO by the 

subscriber: and 

responding to a YES utterance by playing back a stored message. 
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24. The method of claim 4 wherein the step of controlling storage, 
retrieval, and sending of voice messages further comprises the steps of: 

prompting a subscriber to select a save message mode for saving messages 
recently stored on the storage device; 

speaker independent recognition of utterances of YES and NO by the 
subscriber; and 

responding to a YES utterance for saving a recently received stored message 
on the storage device. 

15 

25. The method of claim 20 wherein the steps of controlling storage, 
retrieval , and sending of voice messages further comprises the steps of: 

prompting a subscriber to select an erase message mode for erasing messages 
recently stored on the storage device; 

speaker independent recognition of utterances of YES and NO by the 
subscriber; and 

responding to a YES utterance for erasing a recently received message from 
the storage device. 

25 

26. The method of claim 4 wherein the step of controlling storage, 
retrieval, and sending of voice messages further comprises: 

prompting a subscriber to select a review saved message mode for replaying 
messages previously stored on the storage device by the subscriber; 
30 speaker independent recognition of utterances of YES and NO by the 

subscriber; and 

responding to a YES utterance for replaying a message stored on the storage 

device. 
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27. The method of claim 20 wherein the step of controlling storage, 
retrieval, and sending of voice messages further comprises the steps of: 

prompting a subscriber to select an erase saved message mode for erasing 
messages stored on the storage device by the subscriber in the save message mode; 
speaker independent recognition of utterances of YES and NO by the subscriber: and 
responding to a YES utterance for erasing a saved message from the storage 

device. 

I5 28. The method of claim 17 wherein the step of controlling storage, 

retrieval, and sending of voice messages further comprises the steps of: 

prompting a subscriber to select a send message mode for sending messages 
from the subscriber to one of a plurality of message recipients: 

speaker independent recognition of utterances of YES and NO by the 
subscriber: 

responding to a YES utterance for sending messages from the subscriber to one 
of the plurality of message recipients, comprising the steps of: 

speaker dependent recognition of the name of a message recipient: 
recording a digitized voice message to the recipient on the storage 
device: and 

selectively replaying the voice message upon speaker independent 
recognition of a YES utterance from the subscriber. 

30 29 . The method of claim 28 wherein the step of sending messages further 

comprises the step of enrolling a plurality of speaker dependent templates representing 
digitized utterances of a plurality of names of message recipients. 
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30. The method of claims 18 or 28 wherein the template is created using 
a hidden Markov model having at least fourteen states and having cepstral coefficients 
of at least twentieth order. 

10 

3 1 . The method of claim 30 wherein the step of testing the template 
comprises measuring spectral distance using smoothed Laplacian group delay spectrum. 

32. The method of claim 30 wherein the step of testing the template 
15 comprises step of weighting the cepstral coefficients using smoothed group delay. 
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